critical application
Identifying the Supply Chain of AI for Trustworthiness and Risk Management in Critical Applications
Sheh, Raymond K., Geappen, Karen
Risks associated with the use of AI, ranging from algorithmic bias to model hallucinations, have received much attention and extensive research across the AI community, from researchers to end -users. However, a gap exists in the systematic assessment of su pply chain risks associated with the complex web of data sources, pre-trained models, agents, services, and other systems that contribute to the output of modern AI systems. This gap is particularly problematic when AI systems are used in critical applications, such as the food supply, healthcare, utilities, law, insurance, and transport. We survey the current state of AI risk assessment and management, with a focus on the supply chain of AI and risks relating to the behavior and outputs of the AI system. We then present a proposed taxonomy specifically for categorizing AI supply chain enti ties. This taxonomy helps stakeholders, especially those without extensive AI expertise, to "consider the right questions" and systematically inventory dependencies across their organization's AI systems.
- Oceania > Australia (0.29)
- North America > United States > California > San Francisco County > San Francisco (0.14)
- North America > United States > New York > New York County > New York City (0.04)
- Asia > Taiwan (0.04)
Towards a Trustworthy Anomaly Detection for Critical Applications through Approximated Partial AUC Loss
Bougaham, Arnaud, Frénay, Benoît
Anomaly Detection is a crucial step for critical applications such in the industrial, medical or cybersecurity domains. These sectors share the same requirement of handling differently the different types of classification errors. Indeed, even if false positives are acceptable, false negatives are not, because it would reflect a missed detection of a quality issue, a disease or a cyber threat. To fulfill this requirement, we propose a method that dynamically applies a trustworthy approximated partial AUC ROC loss (tapAUC). A binary classifier is trained to optimize the specific range of the AUC ROC curve that prevents the True Positive Rate (TPR) to reach 100% while minimizing the False Positive Rate (FPR). The optimal threshold that does not trigger any false negative is then kept and used at the test step. The results show a TPR of 92.52% at a 20.43% FPR for an average across 6 datasets, representing a TPR improvement of 4.3% for a FPR cost of 12.2% against other state-of-the-art methods. The code is available at https://github.com/ArnaudBougaham/tapAUC.
An AI Learning Hierarchy
Artificial intelligence (AI) has been successful in numerous areas including speech recognition, automatic classification, language translation, Chess, Go, facial recognition, disease diagnosis, drug discovery, driverless cars, autonomous drones, and most recently linguistically competent chatbots. Yet none of these machines is the slightest bit intelligent and many of the more recent ones are untrustworthy. Businesses and governments are using AI machines in an exploding number of sensitive and critical applications without having a good grasp on when those machines can be trusted. From its beginnings, AI as a field has been plagued with hype. Many researchers and developers were so enthusiastic about the possibilities that they overpromised what they could deliver.
Machine Learning Interpretability and Explainability
Machine Learning (ML) interpretability and explainability are important concepts that refer to the ability of humans to understand and interpret the decisions made by machine learning models. These concepts have become increasingly important as machine learning models are being used in more critical applications, such as healthcare, finance, and criminal justice, where the decisions made by these models can have a significant impact on people's lives. One of the main challenges of ML interpretability and explainability is the complexity of the models. Machine learning models can be very complex, with many layers of neurons and thousands or even millions of parameters. This complexity can make it difficult for humans to understand how the model is making its decisions, which can be a problem when trying to explain the results to non-technical users.
Investigating the Failure Modes of the AUC metric and Exploring Alternatives for Evaluating Systems in Safety Critical Applications
Mishra, Swaroop, Arunkumar, Anjana, Baral, Chitta
With the increasing importance of safety requirements associated with the use of black box models, evaluation of selective answering capability of models has been critical. Area under the curve (AUC) is used as a metric for this purpose. We find limitations in AUC; e.g., a model having higher AUC is not always better in performing selective answering. We propose three alternate metrics that fix the identified limitations. On experimenting with ten models, our results using the new metrics show that newer and larger pre-trained models do not necessarily show better performance in selective answering. We hope our insights will help develop better models tailored for safety-critical applications.
- North America > United States > Arizona (0.04)
- Asia > Middle East > Jordan (0.04)
- Asia > India (0.04)
India Looks To Broad-base AI Workforce To Meet Growing Demand
Only 2.5 percent of engineers in India possess technical skills in artificial intelligence (AI) that the industry requires, the annual employability survey by Aspiring Minds, a job skill assessment firm, found. The World Economic Forum (WEF) estimates that 54 percent of all employees in the Information Technology (IT) space will require significant reskilling by 2022. That demand is especially strong for machine learning and AI. It's not just the working population that is looking to upskill. Retired professionals, school students and homemakers have joined the supply chain, with the definition of learners changing with the coronavirus outbreak. "The most important change that we have seen in the recent past globally, accelerated hugely in the last 18 months, has been a change in the definition of learners.
- Asia > India (0.67)
- Asia > Middle East > UAE (0.15)
- North America > United States > California (0.05)
- (2 more...)
- Health & Medicine > Therapeutic Area > Infections and Infectious Diseases (0.55)
- Health & Medicine > Therapeutic Area > Immunology (0.55)
- Education > Educational Setting > Higher Education (0.35)
- Education > Curriculum > Subject-Specific Education (0.35)
Top 10 global manufacturers using 5G
To further explore the intersection of 5G and manufacturing, register for the 5G Manufacturing Forum. Global manufactuers are starting to adopt 5G to improve manufacturing processes. Low latency and high reliability are needed to support critical applications in the manufacturing field. Several top manufacturers are already taking advantage of 5G implementation to improve operations in different industrial environments. Here we briefly describe some implementations by large manufacturers globally.
- North America > United States > Iowa > Dubuque County > Dubuque (0.15)
- Asia > China (0.05)
- North America > United States > Iowa > Scott County (0.05)
- (9 more...)
- Telecommunications (1.00)
- Automobiles & Trucks > Manufacturer (1.00)
- Information Technology (0.71)
- (2 more...)
Sampling-based Bayesian Inference with gradient uncertainty
Park, Chanwoo, Kim, Jae Myung, Ha, Seok Hyeon, Lee, Jungwoo
Deep neural networks(NNs) have achieved impressive performance, often exceed human performance on many computer vision tasks. However, one of the most challenging issues that still remains is that NNs are overconfident in their predictions, which can be very harmful when this arises in safety critical applications. In this paper, we show that predictive uncertainty can be efficiently estimated when we incorporate the concept of gradients uncertainty into posterior sampling. The proposed method is tested on two different datasets, MNIST for in-distribution confusing examples and notMNIST for out-of-distribution data. We show that our method is able to efficiently represent predictive uncertainty on both datasets.
- North America > Canada > Quebec > Montreal (0.04)
- Asia > South Korea > Seoul > Seoul (0.04)